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Abstract 

Experiencing artworks in the Web aggravates the problem of the absence 
of any historical, cultural and social context, because the Web is a 
veritable nowhere. This paper stresses the importance of a social and 
narrative access to art, arid presents an interactive group of virtual 
characters as an alleviation of this problem. The benefits and overall 
concepts of this approach are examined. Particularly, the importance of 
guiding the virtual characters to establish close emotional relations with 
the user for achieving narrative immersion is stressed, and ways of 'v 

accomplishing this task are sketched. 
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Introduction 

Viewing artworks on the Web is an experience absent of any historical, cultural, 
or social context. While this problem is well known to traditional museums, the 
Web, as a veritable nowhere, aggravates the situation. Consider someone 
looking at a Byzantine icon at a Web museum with a home monitor. A visitor — 
henceforth the "user"- not very well acquainted with the subject is likely to miss 
much of the meaning of the artwork, and the art experience will be shallow and 
uninteresting. 

It is certainly helpful to offer some additional written explanation or even video 
material, but still, the experience remains intellectual and solitary, rather than 
emotional and cultural enrichment. Additionally, it is not easy to motivate the 
visitor to study the additional material, instead of clicking away after some 
superficial glance. 

What can be done to provide a more involving and enriching art experience to the 
user? According to the European hermeneutical tradition, understanding of an art 
work is inherently a social, dialogic and emotional process (Cf. Gadamer, 1993; 
Heidegger, 1973). There is no absolute meaning, but only restricted perspectives 
that belong to a whole live world. In order to understand a Byzantine icon, a user 
must understand the artists who made it and the worship of the believers, their 
way of living, their feelings, and their convictions. This is not simply a matter of 
increasing a person's knowledge, but to fuse his own horizon with the horizons of 
the people to whom the icon had an essential meaning. This is not only an 
intellectual but also an emotional process of feeling empathy and identifying with 
particular individual motivations and biographic experiences. 



At this point, narration enters the scene as a means of promoting identification, 
empathy and understanding. Through stories, it is possible to boost the fusion of 
horizons in art experience. Examples of this process in traditional media are 
abundant, e.g. historical novels or films. 
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A new approach is "interactive storytelling". Interactive storytelling is the 
discipline that deals with the run time creation and adaptation of stories, 
observing emergent constraints like global duration of the interaction or user 
choices and interests. It aims at combining the narrative immersion of traditional 
story telling with the advantages of user behaviour adaptation, employing, for 
example, educational strategies. 

In this paper, I will present a narration driven, interactive group of virtual 
characters with whom the user can fraternize and discuss each other's art 
experience and perspectives. This group allows reconstituting the contingent 
meanings of art and regaining the missing emotional and existential perspectives 
on it by preserving the narrative horizons. At the same time, the group involves 
the user through interaction and adaptation. 




Figure 1 — Two virtual characters speaking about a religious icon 

This interactive group is being developed in the project Art-E-Fact (Cf. www. art-e- 
facborg ). This EU-funded project, a collaboration of several institutions of four 
European countries, aims at developing generic tools for artists to create dialogic, 
narrative worlds inhabited by personality-rich virtual characters. The interaction 
with these worlds is enhanced by specialized utilities like exploration tools that 
allow the user to uncover different painting layers. Though the project does not 
aim specifically at Web based use, incorporating an internet version poses no 
technical problem. A VRML viewer, together with some small plug-in and a freely 
available speech synthesizer, are the only technical necessities. The initial 
demonstrator deals exactly with a museum presentation of the Byzantine icons 
mentioned above (Cf. www.monastery-artdiagnosis. gr for a link to one partner). 

The next section gives an overview of previous work. Then, I will present the 
overall architecture of the system. After that, I will explain the benefits and some 
ideas behind the project. I will stress especially the importance of automating the 
process of creating social and emotional bonds with the virtual characters to 
foster the experience of narrative immersion. 

Related work 

The importance of narration for the structuring of experiences is also stressed in 
psychology, e.g. in the work of Shank and Abelson (1995) and Graesser and 
Ottati (1995). 

Andr6 et al. (2000) developed a non-interactive presentation group without 
narrative structures. Their emphasis lies in natural language generation 
independent of personality traits of the virtual participants. Mateas and Stern 
(2000) are developing a very promising system for interactive storytelling. In 
contrast to our approach, they do not aim at education and presentation, which is 
probably the reason for the different architectonic and conceptual details, since 
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there is no need to present information in their system, and therefore a less 
centrally guided approach is possible. 

Rickel et al. (2002) are engaged in an ambitious project that comprises natural 
language interaction, emotional and social simulation, but not information 
presentation and narration. A. Paiva, I. Machado and others have developed 
various interactive applications with storytelling and/or tutorial components (cf. 
Machado etal., 2001, Paiva, 2001), though none of these systems uses 
interactive discussion for multifaceted edutainment. 

Another project in which our own group is involved is "Geist" (cf. 
www.tourgeist.de ). where interactive storytelling is used, but without any natural 
language interaction , and "Virtual Human", which aims at the creation of a 
narrative tutoring system with talking virtual characters (cf. w ww.virtua l- 
hu man.or g). 

Architecture of the System 




Figure 2 — The basic architecture 

The basic architecture is depicted in figure 2, The information process is 
centralised and hierarchic, the flow of control starting at a story engine and 
passing through a scene engine to the dialogue manager or directly to the scene 
player. The player executes the scripts delivered either by the scene engine or by 
the dialogue manager and controls the partially autonomous virtual characters 
and objects. The current architecture separates strictly interactive and non- 
interactive episodes. In an interactive episode, the user can talk to the virtual 
characters using a keyboard, and the dialogue manager is responsible for the 
generation and distribution of adequate answers. In the case of a non-interactive 
episode, the scene engine controls the virtual characters directly. Non-interactive 
episodes are fixed sequences authored in advance, much like the dialogues in a 
traditional theatre play. 

Story Engine 

Our basic approach to the automated generation of stories is to use a central 
narration component, called "story engine", that combines and adapts 
prefabricated minimal story components during run time, according to rules e.g. 
of narration or pedagogy, and to the interaction results. Those rules are stored as 
story models. The minimal story components are called "scenes". For details of 
this process, compare Braun (2002), Spierling et al. (2002). The scene engine 



o 

ERIC 



BEST 



file://E:\mw2003\papers\iurgel\iurgel.html 



5/27/2003 



plays the scenes that were chosen and adapted by the story engine controlling 
the subordinate modules. 



Dialogue Manager 

The dialogue manager is granted control by the story engine if an interactive 
episode is part of a scene. It is centralized in order to allow using distributed 
reactions, where, for example, two virtual characters mention two different 
aspects of a problem as an answer to some question of the user. Its output is a 
script to the dialogue manager. The script may consist of a single utterance or of 
a series of commands to the subordinate modules. Currently, the dialogue 
manager relies on a modified version of the chatbot ALICE (Cf. www.alicebot.org ) 
which makes use of a "natural languagee processing database” (NLP-DB). The 
use of ALICE is only a provisory solution until some more powerful system is 
developed. 

Scene Player 

The scene player executes scene scripts. The scripts control the appearance of 
the characters and their behaviour at varying levels of abstraction, and govern 
the objects in the scene; for example, the appearance of icons or the starting of 
films. 

Object Manager 

The object manager is controlled directly by the scene engine. It manages the 
display of the objects of the scene; for instance, the appearance and 
disappearance of the artworks under discussion. 

Virtual Characters 

The virtual characters are responsible for the generation of consistent behaviour 
according to their personality traits and situation appraisal. This behaviour 
includes turn-taking signals and conversational feedback as well as emotional 
reactions and goal management. The virtual characters are composed of a mind 
called agentive person, a simulation of body processes called bodily person, and 
a graphics output module called manifest person. The parameters of these 
personality aspects work together to create the final visible personality of the 
virtual character. The prescriptions may vary in their level of abstraction, ranging 
from detailed commands of movements to mere annotations in the sentences to 
say, informing the characters about the pragmatic meaning of the sentences. In 
the last case, every detail of the performance is in the charge of the virtual 
characters. Thus, their function is analogous to actors in a theatre play, 
interpreting with leeway the instructions of a director according to their characters 
and situation awareness. For more details, cf. lurgel 2002. 

One of the central research goals of the project consists in understanding how to 
combine story elements prepared in advance with less guided, highly interactive 
episodes. Both aspects are necessary in interactive storytelling, since only the 
use of prepared dialogues and instructions guarantee an interesting and 
consistent storyline, and only the introduction of free interaction enables user 
adaptation and involvement. 

As in any story, the main responsibility for success and suspense lies in the hand 
of an author. Story creation without an author — "emergent narrative” — is not 
likely to provide the intended quality of emotional experience. For this reason, in 
the project "art-e-fact”, artists are involved in every step of the development. 
Thus, suitable authoring tools for interactive storytelling are vital and one of the 
main challenges of the project (Cf. Scheider, 2003).Ways of Fostering Art 
Understading. Fostering art understanding through the use of a virtual 
conversation group. 
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In this section, I want to elaborate more on some aspects of using a virtual 
conversation group to promote the understanding of art works. In order to 
concretize the discussion, I introduce two scenarios: 

1. Distant horizon scenario. 

In this scenario, the virtual characters are contemporaries of the human 
participant and maintain themselves at a distant perspective from the art work, 
though it has an important meaning for their lives. For example, a scientist who 
analyses materials and colours, and an historian who knows about cultural 
context, might be present. This scenario is a good starting point for the 
participant, because it is easy to understand the virtual characters, and the 
interest of the characters in the icon can spread to the user. 

2. Near horizon scenario. 

Here, the virtual characters live in the past and share the religious fervor that 
once accompanied the icon. A monk and a mother of a sick child might be 
present. This constellation must be introduced more carefully, but promises a 
more intense experience, involving more central feelings like suffering, 
compassion, hope, and so on. 




Figure 3 — The virtual characters as talking heads 

Keeping these scenarios in mind, I will now mention some advantages of using a 
conversation group to present the art work on the Web: 

1. Perspective. Every character can support a definite perspective, e.g. a 
historian and a scientist. This allows elucidating the theme from different, possibly 
incommensurable points of view. Personality trait differences have here the 
important function of establishing a link between opinion and emotion. For 
example, a reserved and sceptical character is suitable for the role of a scientist, 
but probably not for the horizons of a religious person. 

2. Dialogic deployment. The use of a group of characters facilitates the dialogic 
deployment of thoughts, because insightful sequences of questions, contributions 
and answers can be prepared in advance. Even if the user is included in the 
deployment, the presence of a group is technically and conceptually 
advantageous, because one virtual character can provide the keyword needed by 
the other to continue the dialogic developments. 

3. Narration. It would be very difficult to maintain a narrative structure using only 
a single virtual character because of the unpredictability of user behaviour. If, for 
example, the author prepares a narrative structure where the character 
necessarily has to speak about his early childhood experiences, but the user 
refuses to keep at it, the story line is likely to fail. This cannot happen if there is 
another character willing to listen in the place of the user. 
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4. Natural language understanding. Communicating with a single virtual 
character is difficult simply because it very often won't understand the user. The 
use of a group alleviates the problem, because it is quite natural that some 
contributions do not receive a proper answer in a discussion group. Shortcomings 
of natural language processing do not lead to a breakdown in the conversation. 

5. Social processes. Social and group processes are among the strongest 
elicitors of emotional arousal. According to Kemper (1991), there are only two 
basic dimensions that govern the arousing of emotions in a social relationship; 
namely, "status"and "power". Those dimensions can be exploited to generate 
emotionally thrilling plots involving the virtual characters and possibly including 
the user. For example, the user might be exposed to a planned power change in 
the course of the narration, his influence as a group member increasing or 
decreasing depending on his behaviour and on the plot. This can be achieved by 
a sequence of scenes in which the characters initially disregard the suggestions 
of the user; in the course of the interaction, his contributions would be 
"recognized"by the virtual characters as important and increasingly mandatory. 

The nowadays prevalent use of anthropomorphic agents on the Web considers 
only the usage of a single character. 1 argued that this is a disadvantageous 
restriction that impedes narration and weakens the interactive experience. 

"Interactivity"in interactive storyteiiing 

A non-interactive example for the combination of story and presentation is 
"Sophie's World" (Gaarder, 1996), where the adventures of a girl serve as a 
background for explanations in the history of philosophy. Here, the figure of 
Sophie is used to promote identification, gradual presentation of knowledge, and 
emotional involvement in the theme. 

Certainly, stories in interactive storytelling currently are at risk of not reaching the 
narrative quality level of non-interactive media where every detail can be 
controlled by the author or director. Because of this, understanding the role of 
interactivity in interactive storytelling remains an essential and still not 
satisfactorily accomplished task. What are the benefits of introducing interactivity 
into storytelling? Of course, the conflicting goals of the application provide good 
reasons, even if it should turn out that the intensity of narrative immersion is 
affected. Some examples are: 

1. Navigation. In an edutainment application, it is important that the user can 
navigate; for example, in order to go back to themes already discussed. 

2. Reviewing and testing. Interaction is necessary for feedback about some 
educational goal, e.g. the acquired knowledge could be tested. 

3. Joy of expression. Expressing personal opinions and observing the virtual 
character's reactions is a joyful experience. 

4. Exploration. The user can be engaged in explorative learning strategies, e.g. 
a virtual character can gradually guide the user to discover more details of an 
icon. 

However, the starting point of this paper was the social and emotional nature of 
art understanding and the importance of narrative immersion in this context. The 
question at hand is whether it is possible to use interactivity to intensify the 
experience of narrative immersion. Traditionally, emotional involvement in a story 
in achieved by identification with a protagonist. The prospect of placing the user 
in the protagonist's position in interactive storytelling might be thrilling, but would 
be opposed to the current goal of rendering another person's emotional and 
cultural perspective understandable — aside from the technical constraints. 

Therefore, I follow the indirect path of creating planned emotional proximity — 
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henceforth "friendship" — between the user and a chosen protagonist virtual 
character. The reason for this is that friendship is known to boost identification 
and empathy, and thus, should increase narrative immersion. 

An example is a virtual character that is emotionally touched by discovering, in 
the course of an adventure, the symbolic meaning of a religious icon. Because of 
the establishment of some sort of friendship with the character, his emotional 
reaction should also touch the user, promoting his understanding and 
appreciation of the icon and of its religious context. 

Improving Art Understanding Through Friendship With Your Virtual 
Companion 

Previously, I argued that it is possible to foster the narrative immersion through 
the establishment of some sort of friendship between the user and the virtual 
character, thus intensifying the identification with the character and his emotions. 
In pedagogy, those phenomena of learning facilitation under the influence of a 
liked person fall under the broad and well studied heading of "social 
learning" (Bandura, 1977, is a classical source). In other words, the idea 
presented here is that "social learning"is also useful when the main channel is a 
narration. The research issue underway, now, is to automate the creation of 
friendship between character and user. In the present context, the relevant 
psychological factors that determine friendship ("attraction") are the following (Cf. 
Fehr, 1996, for an overview of those factors): 

1. Similarity. There is plenty of empirical evidence that similar character traits 
and opinions are favourable to the establishment of close emotional relationships. 

2. Reciprocity.Basically, reciprocity means that we tend to like someone who like 
us. This dimension is particularly sensitive to temporal factors. For example, 
increasing sympathy over time is more favourable to the establishment of 
emotional proximity than decreasing it, even if the final sympathy level remains 
the same. 

3. Self-Disclosure. Self-disclosure is an important indicator of friendship. A 
person who discloses his feelings to me probably likes me. 

4. Maximized gains in social exchange. Praise is also very favourable for the 
establishment of friendship. For example, it is advantageous for me if another 
person praises my doings, thus raising my status and my self-esteem. 

5. Common enemy. The existence of a common enemy is also favourable for 
the establishment of camaraderie as a variety of friendship. 

Every one of those factors can be incorporated into a story-telling system. For 
example, the virtual character chosen for identification can be made to mimic 
similarity ("You do not like this icon? Nor do I!"); reciprocity ("That’s fine that you 
are enjoying our conversation! I am also learning very much from your 
contributions!"); or self-disclosure ("I am full of anguish when I think that my 
answers might not match your questions.") This can already be achieved by 
integrating an appropriate plot with fairly simple extensions of ALICE, which allow 
the story engine to guide this chatbot. More powerful methods for interactive 
episodes, based on the state-update-theory of dialogue moves, are under 
investigation. 

The details and distinctions are tricky and the object of current research. 
Therefore, the next section will elucidate more the process of automated creation 
of friendship in interactive storytelling by presenting an example. Example 

In this example, it is assumed that an appropriate story is generated and 
envelops the scenes under discussion. An abstract initial scheme for the 
automated establishment of friendship is depicted in figure 4. It is assumed that 
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"discovering similarities", "taking sides", "self-disclosure", and "conflict 
resolution"are scenes controlled by the story engine according to friendship 
establishment models. 
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Figure 4 — Possible steps of automated friendship forming 

The creation of this model is a process that involves computer science, 
psychology, artistic intuition and narrative skills. The scene "taking sides"is 
included because it allows the virtual character to show how his emotional well- 
being depends on the behaviour of the user, thus expressing the close friendship 
he presumes to have with the user. At the same time, the scene "taking 
sides"creates a common enemy. Short interaction examples follow, in which the 
system tries to establish friendship between Virtual Character 1 and the user: 

DISCOVERING SIMILARITIES 

Virtual Character 1: "How do you like this icon?" 

User: "I love it!" 

Virtual Character 1: "I love it tool" 

(The virtual character agrees with the user, no matter what he 

says.) 



TAKING SIDES 

Virtual Character 2 (to Virtual Character 1): You talk too much 
nonsense!" 

Virtual Character 1 (shocked, to User): "Do you think, too, that I 
talk nonsense?" 

User: "Not at all!" 

Virtual Character 1 (showing relief and gratitude): "You are a real 
friend!" 
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(Virtual Characters 1 has now a slot to show his deep feelings, 
depending on the side the user takes. Virtual Character 2 
becomes antagonist of Virtual Character 1. The scene would have 
to be extended by the story engine if the user had agreed with 
character 2, in order to resolve amicably the emerging conflict 
between user and character 1.) 



SELF DISCLOSURE 

Virtual Character 1 (to User. A monk has appeared in the scene.): 
"I am afraid of this monk. Do you know why?" 

User: "No.- 

Virtual Character 1: "Because he resembles my father!" 

(It is important here to take care that the self-disclosure be not 
ascribed to personality traits of the virtual character, but to his 
special relationship with the user.) 



RESOLVING CONFLICT 

Virtual Character 1 (to Virtual Character 2. Virtual Character 1 is 
still angry at 2): "Do you still think that I am talking non-sense?!" 

Virtual Character 2: "Oh, you are still angry at me. No, please 
excuse me, l was wrong." 

Virtual Character 1: "OK, let us forget about it." 

(The conflict was only introduced for the sake of approaching 
Virtual Character 1 and the user. Since a latent conflict in the 
group might be harmful to the further development of the story, it 
is better to resolve it soon.) 

Conclusion 

l presented the project Art-E-Fact and the use of a narrative, interactive 
discussion group to deepen and contextualize art presented on the Web. I 
referred to art theories which state that art understanding is an inherently social, 
emotional and cultural process, and that therefore narration and dialogue are 
appropriate means of fostering art experience. I presented an overall initial view 
of the system and argued that creating friendship with the virtual companions is 
likely to improve the process of identification and narrative immersion, and 
sketched the automated processes that establish the desired emotional proximity. 

The ongoing issue might be regarded as a first attempt to virtually populate 
spaces that were until now wasteland like the Web, providing moments of 
automated suspense, immersion, and joyful learning to the interested user. 
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